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ABSTRACT 

This study investigated whether preservice teachers* self- 
ratings of their skills and knowledge were comparable to the ratings of the 
same set of skills by their closets faculty members, in this case, their 
assigned mentor leaders. The participants were eight cohorts of students 
receiving their initial teacher certification and about to graduate. The 
eight mentor leaders had served as advisors for these students for their 
junior and senior years in most cases, although the two graduate-level 
cohorts had only been together for one school year in secondary education. 
Both groups completed identical Likert-type questionnaires, and the Spearman 
Rho was used to correlate the sets of answers. Results were as expected for 
the most part; students rated themselves higher than the mentor leader did. 
The best correlation was found in the estimation of technology skills, 
ability to choose teaching materials, and the ability to design teaching 
units. Correlations for leadership and ability to plan lessons were also 
statistically significant, although lower than expected. A secondary analysis 
comparing the percentage of just ratings of excellent was done for the set of 
questions. Three of the mentor leaders had moderate to high correlations with 
their students' self-ratings for the set of abilities. Since 93% of the 
students regarded their overall ability to begin teaching as good or 
excellent, and 93% of their mentor leaders felt the same, students' responses 
can be regarded as valid, and it can be expected that they will have a 
successful first year of teaching. With regard to specific abilities, the 
mentor leaders could stress more self-evaluation or collect more data about 
the lowest correlated skills and abilities. An appendix contains the forms 
used in the study. (Author/SLD) 
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Abstract 



The purpose of this study was to determine if our preservice teachers’ self-ratings of their skills and 
knowledge were comparable to the ratings of this same set of skills by their closest faculty member, in 
this case their assigned mentor leader. The participants were eight cohorts of students receiving their 
initial teacher certification and graduating at this time. The eight mentor leaders served as advisors for 
their group of student for both their junior and senior years in most cases. In the case of the two 
graduate level cohorts, the program was only for one year in secondary education. The mentor leaders 
also served as instructors for various (but not all) of the courses taken by their assigned students. Four 
of the mentor leaders were new to the school of education and this was their first cohort. The 
questionnaire had 13 questions related to the stated goals of the school of education in their conceptual 
framework as well as an overall readiness to begin teaching question. Both groups completed identical 
versions by circling the Likert scale elements. The forms were matched by student identifying numbers 
but were otherwise confidential. The data analysis used the Spearman Rho to correlate the sets of 
answers. The results were as expected in that for the most part; the students rated themselves higher 
than their mentor leader did. The best correlation was found in the estimation of technology skills (Rho 
= .34), ability to choose teaching materials (Rho = .3 1), and the ability to design teaching units (Rho = 
30). Correlations for leadership (Rho = .27) and ability to plan lessons (Rho = .27) were also 
statistically significant. However, these correlations were lower than expected. This is partly due to the 
restriction in the range of the answers since most of the ratings were either good or excellent. Therefore 
a secondary analysis comparing the percent of just the ratings of excellent was done for the set of 
questions. Three of the mentor leaders had moderate to high correlations with their students’ self-ratings 
in the set of abilities. Two were experienced and had several sets of cohorts to graduate. The third was 
one of the new mentor leaders. Since 98% of the students regarded their overall ability to begin 
teaching as good or excellent and 93% of their mentor leaders felt the same, the students’ responses can 
be regarded as valid, and we expect them to have a successful first year of teaching. In regards to 
specific abilities, the mentor leaders could stress more self-evaluation or collect more data in regards to 
the lowest correlated skills or abilities. 
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Student Self-Evaluation of 



Teaching Abilities at Graduation from GC&SU 

Teacher Education in the Initial Preparation Program 

The purpose of this study was to investigate whether the self-ratings that the exiting seniors gave 
themselves on the 13 teaching skills/abilities emphasized in our conceptual framework would be 
comparable to the ratings given by them to their mentor leaders. It was expected that the seniors would 
probably rate themselves somewhat higher than their professors would, but that there would be a 
positive correlation of these ratings. 



Methods 



Participants 

There were a total of 108 students and 8 professors participating in the study. There were 6 
cohorts from the main campus at Milledgeville and 2 cohorts from the satellite campus at Macon State 
College. The Milledgeville cohorts were Early Childhood (n = 27), Health and Physical Education (n = 

7), Middle Grades (n = 16), Music (n = 4), Special Education (n = 19), and Secondary Education (n = 

15). The Macon cohorts were Early Childhood (n = 10) and Secondary Education (n = 8). The mentor 
leaders were all professors who served as advisors and course instructors in the major field. The cohorts 
were students who started their professional education coursework at the start of their junior year and 
took all their classes together for the two year period culminating in their student teaching during the 
spring of their senior year. 

Instrumentation 

One page questionnaires were used. See Appendix. The professors were instructed to rate each 
student in their group (cohort) at the end of his/her student teaching. “How would you rate his/her 
present abilities as a beginning teacher?” The students were asked “How would you rate your skills as a 
beginning teacher right now?” The 13 teaching skills were described and the rating scale was excellent, 
good, fair, or poor. There was an additional question (which was actually question one) in which the 
person was asked to rate their ability to “begin teaching overall”. The last five digits of the students’ 
social security numbers were used to match the questionnaires for data analysis. 

Procedures 

The students were given the one page questionnaire during a session in which they all met with 
the SOE certification officer to complete their paperwork for applying for teacher certification during 
their last month at GC&SU. The mentor leaders were given a comparable version of the form at about 
the same time and asked to rate each of their graduating seniors using the same rating scale. The 
instructions on the forms stated that the participants were to base their ratings or self-ratings on their 
skills/abilities as a beginning teacher. 

Data Analysis 

This was a descriptive study and Spearman Rho correlations were used. Data were analyzed as a 
whole and then cohort by cohort. 

Results 

Most of the students (97%) rated themselves as Excellent or Good on their overall readiness to 
begin teaching and on most of the different teaching skills. The exception was the MAT cohort on the 
main campus who rated themselves somewhat lower. The mentor leaders also used mostly Excellent or 
Good (93%) to rate their students as beginning teachers. The proportion of Excellent ratings was much 
higher for the students than for the mentor leaders which we expected to find. There were some positive 
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and statistically significant correlations of the student and mentor leader ratings on about half of the 
skills rated. Kghest correlated were the ratings for the use of technology, evaluating teaching materials 
and planning lessons. The actual Spearman Rho correlation coefficients were quite low (.33-.30). This 
is probably due to the restriction in range of the ratings (dependent variables) since most of the ratings 
were either a 3 (good) or a 4 (excellent). 

Then the percent of ratings of excellent were calculated for both the students and the mentor 
leaders for all of the 14 skills listed, actually 13 skills and one overall teaching readiness rating. When 
these numbers were correlated as a set, the value of the correlation was much higher (r = .52, n = 14, p = 
.03, one-tailed). Three of the mentor leaders got moderate or higher statistically significant correlations 
of the ranking of their students’ perceived skills at exit from teacher preparation. The HPE and Music 
cohorts were so small in number (less than ten) that their numbers are suspect for true validity. Scatter 
plots are included in this report, both for the dataset as a whole and individually by cohorts that illustrate 
the aspects of teaching where the agreements and disagreements are by quadrants. As expected the 
mentor leaders’ ratings are lower than the students’ self-ratings in most aspects, including their overall 
readiness to begin teacher. 

Then the difference between the mentor leaders’ and students’ ratings of excellent in percents 
was calculated. Most of the mentor leaders differed from their cohorts on the various skills only about 
plus or minus 20% in their ratings of excellent. A set of bar charts accompanies each scatterplot and 
shows the discrepancies from overratings by the mentor leaders to underratings by the mentor leaders 
(from the student’s point of view!). 

Discussion and Conclusions 

This study validated (for the most part) the self-ratings of the teacher education candidates and 
the mentor leaders. The JHL-SOE produces mostly excellent or good beginning teachers. This was also 
the finding in the employer survey conducted three years ago. Of the eight cohort leaders who 
participated in this study, only 4 were experienced. The other four were new and this was their first 
cohort to complete the GC&SU teacher education program. Our program has a unique conceptual 
framework for teacher education with a large field-based component and many other additional 
requirements designed to enhance the ability of these students to work cooperatively with others and 
develop their liberal arts skills; especially skills in communication, problem solving, leadership, and 
self-evaluation. 



Results and Figures 

Figure 1 - Comparison of Ratinas of Overall Ability to Begin Teaching 
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These differences were statistically significant at the p < .01 level. 

Figure 2 - Percent of Ratin 2 S of Excellent by Major 
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The correlation between the students’ and mentor leaders’ of ratings are statistically significant and 
positive, but quite low (Rho = .20, n = 103, p = .02). The low value of the correlation may be due to the 
restriction in range of the values, since most of the ratings were either 3 (good) or 4 (excellent) by both 
the student and the professor. 
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The correlation of the students’ self-ratings and the ratings by their mentor leader is highest for using 
technology in teaching. The table below also shows the percent of the ratings of Excellent on a question 
by question basis for both the students and the mentor leaders. Only the five highest correlations are 
statistically significant. 

Table 1 — Item bv Item Correlations and Percent of Students Rated “Excellent” 



# 


Question 


Correlation 


Students 


Mentors 


3 


Technology 


.34 


35 


29 


13 


Evaluate Materials 


.31 


64 


49 


6 


Design Units 


.30 


48 


40 


14 


Leadership Role 


.27 


44 


42 


5 


Plan Lessons 


.27 


62 


39 


4 


Manage Behavior 


.22 


31 


25 


1 


Begin Teaching 


.20 


58 


39 


7 


Assess Learning 


.21 


33 


27 


11 


Self-Evaluate 


.19 


49 


50 


9 


Ask Questions 


.15 


34 


52 


8 


Teach Diverse Students 


.13 


40 


57 


2 


Content Area 


.13 


38 


25 


12 


Ethics & Laws 


.12 


67 


60 


10 


Listen & Respond 


.10 


50 


58 



Fi£ure 3 - Differences in the Ratings of Excellent are Plus or Minus 20% 
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Difference in Percents 



The mentor leaders rated the students higher in ability to Ask Questions and lower in Planning Lessons. 
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However when the percent of ratings of excellent for the students and the mentor leaders are correlated, 
the correlation between these ratings is higher than for any individual variable (r = .52, n = 14, p = .03 
one-tailed). See Scatterplot below. 

Figure 4 - Scatterplot of Item bv Item Correlations 




Mentors 



The most discrepancies are in the upper left and lower right quadrants. The upper left quadrant has 
aspects where the students’ self-ratings were much higher and the lower right quadrant shows where the 
students” self-ratings were much lower than the mentor leaders’ ratings. 
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Appendix 



The first form is the mentor leader version. This is followed by the instructions. 

The second form is two pages which were separated. Only the first page was used for this study. 
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John H. Lounsbury School of Education 

Teacher Preparation Program Evaluation Validity Study 

Spring 2002 



Please rate each student in your cohort at the end of her/his student teaching. 



Student SSN 

How would you rate his/her present abilities as a beginning teacher? 
1. begin teaching overall 


{Please circle your answer) 
Excellent Good Fair Poor 


2. develop his/her content area knowledge /reading, math, science, etc.) 


Excellent Good Fair Poor 


3. use technologv in his/her new teaching job. 


Excellent Good Fair Poor 


4. manage classroom behavior of students. 


Excellent Good Fair Poor 


5. plan dav to dav lessons. 


Excellent Good Fair Poor 


6. design teaching units. 


Excellent Good Fair Poor 


7. assess the learning of their students /teaching effectiveness). 


Excellent Good Fair Poor 


8. understand diverse learners and find the strengths in each student. 


Excellent Good Fair Poor 


9. tactfiillv ask the right Questions to get information from students, parents 


coworkers and supervisors. 


Excellent Good Fair Poor 


10. listen and respond professionals to parents, students, and coworkers. 


Excellent Good Fair Poor 


11. self-evaluate his/her work and decide how to improve next time. 


Excellent Good Fair Poor 


12. understand and complv with professional ethics and educational laws. 


Excellent Good Fair Poor 


13. evaluate and find good teaching materials and resources. 


Excellent Good Fair Poor 


14. olav a leadershio role in his/her school or communitv. 


Excellent Good Fair Poor 


Thank you for helping in this study. Please give all of these evaluations in a batch to Sharie Smoot with your name, total 


number of students, with major and campus. 





March 18, 2002 



From: Sharie Smoot & Program/ Assessment Committee 

To: Mentor Leaders of Exiting Seniors and M. A.T. Initial Preparation Programs 

Re: Validity Study of the Exit Survey 

This year only, we are going to compare the student’s self-ratings on the various aspects of our 
conceptual framework with the mentor Leader’s evaluation of that same student. Some time in April, 
please complete one of these rating forms for each student. Just put the student’s social security number 
on the top and circle your rating for each student. Keep in mind that we have asked them to rate their 
skills as a beginning teacher on a form identical to this one. 

We expect to find a positive correlation between your ratings and their self-ratings of their skills as a 
beginning teacher. Please be honest and just circle Excellent, Good, Fair (OK) or Poor for each question 
in regard to that particular student. 

Don Jackson will be giving this same survey along with a satisfaction survey (that part will be 
anonymous) when he gets the students together for doing their certification paperwork. 

When you are finished, please bring your forms/surveys all together as a batch and give them to me with 
your name, total number of students, with major and campus. 
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John H. Lounsbury School of Education 
Teacher Preparation Program Self-Study Spring 2002 



How would you rate your skills as a beginning teacher right now? 



1. ready to begin teaching overall ^ 

2. your content area knowledge (reading, math, science, etc.) 

3. using technology in your teaching job. 

4. managing classroom behavior of students. 

5. planning day to day lessons. 

6. designing teaching units. 

7. assessing the learning of your students (your teaching cflFectivcncss). 

8. understand diverse learners and finding the strengths in each student. 

9. tactfully asking the right questions to get information from students, parents 

coworkers and supervisors. 

10. listening and responding professionally to parents, students, and coworkers. 

11. self-evaluating vour work and deciding how to improve next time. 




(Please circle your answer) 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
_Excellent Good Fair Poor 

_Excellent Good Fair Poor 
_Excellent Good Fair Poor 
Excellent Good Fair Poor 



12. understandina and complying with professional ethics and educational laws.E xcellent Good Fair Poor 

13. evaluating and finding £Ood teaching materials and resources. ^Excellent Good Fair Poor 

14. playing a leadership role in vour school or community. ^Excellent Good Fair Poor 



Your answers will be kept confidential, but we want to be able to analyze your answers in various ways to help improve our 
program. By giving us Ae last five digits of you social security number, it saves us from asking you a lot more questions. 

The last five digits of my SSN are XXX - X — 

(Please go on to the next page, when you are done, tear these two pages apart and turn them in separately so that the next 
part can be anonymous) 



This part is about the field-based^ cohort process that you participated in. Please rate: 



15. The suDDort vou received from vour cohort fmentor) leader. 


Excellent Good Fair Poor 


16. The nrofessional & theoretical knowledge of vour instructors here. 


Excellent Good Fair Poor 


17. The preparation vou received for the Praxis II (licensing) exams. 


Excellent Good Fair Poor 


18. The way your instructors helped you make connections between your 




classwork and vour fieldwork. 


Excellent Good Fair Poor 


19. Most of vour field exoeriences in the local nublic school classrooms. 


Excellent Good Fair Poor 


20. The teaching ability of most of vour field olacement (host) teachers. 


Excellent Good Fair Poor 


21. Overall, vou host teachers’ ability to helo vou imnrove vour teaching. 


Excellent Good Fair Poor 


22. The core courses vou took at GC&SU (leave blank if vou transferred here). 


Excellent Good Fair Poor 


23. Mv cohort was Middle Grades Early ChildhoodSpecial Education 


M.A.T. 



Music Health/Physical Education 



24. The campus I attended was Macon Milledgeville 

There were some really good aspects of my teacher preparation program at GC&SU. They were: 

1 . 



2 . 



3. 



There were some aspects of the teacher preparation program at GC&SU that could be better. They are: 

1 . 



2 . 



3. 



Thank you very much. We appreciate the continued involvement of our graduates. Without hearing 
from you and our alumni, GC&SU would not be the quality teacher preparation program that we feel it 
is today. Best wishes in all aspects of your life! 
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